Papers with multimodal self-attention
Multimodal Transformer for Multimodal Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods to incorporate information from other modality, usually static images, are not considered relative to multimodal machine translation. |
| Approach: | They propose a multimodal self-attention method which learns the representation of images based on the text, which avoids encoding irrelevant information in images. |
| Outcome: | The proposed model outperforms previous studies and competitive baselines in terms of various metrics. |